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Abstract 

The current study reports a two-year research project funded by the Government of the Republic of Indonesia 
through a competitive research scheme. The aim is basically to respond to the fact most university students have 
very low interests in reading activities, such as finding out important information for their term papers as 
assigned by the lectures. Instead, most of the time is spent in BBM, IPhone chats and Facebooking of 
non-academic nature (mostly social encounters). This has triggered a team of researchers to find out ways to 
increase or encourage reading interests. Internet browsing was undertaken to search for possible software 
application systems which could be used to administer online assessments in reading class. It was Question 
Writer (QW3.5) selected for use in the current study. It is a paid software application system especially 
developed for online assessments. It can perform various types of question formats with the students’ responses 
directly forwarded to the teacher’s email, and feedbacks and scorings automatically performed by the system. In 
the current study, a discussion group called ‘Reading Maniacs’ was created in Facebook for the students to get 
access to both reading materials and assessments. Questionnaire and interviews were conducted to investigate 
how the students got motivated in reading class and if their reading interests got increased. The findings indicate 
that the students were very motivated to participate in the online assessments supported by Facebook group 
discussion, thereby their reading interests leveled up. It was therefore recommended that online assessment of 
reading skills be conducted as additional activities to the well-supervised offline reading examination. Future 
researchers may want to administer Questionnaire to the reading teachers to get some feedbacks. 

Keywords: online assessment, reading interests, software application, facebook group discussion 

1. Introduction 

1.1 Background 

English language teachers or possibly any other teacher of humanity disciplines have worried a lot over the low 
level of reading interests. It was claimed (Kusmana, 2009) that Indonesia was at the lowest level of reading 
interest among other East Asian countries—only 30% of the text can our students absorb. This represents a 
warning that our students are of low literacy level. Meanwhile, it is commonly realized that reading is one of 
language skills that has been introduced at the beginning of language learning period. Reading is important for 
all ages, (Harmers, 2003, p. 190 ) states that reading is an exercise dominated by eyes and brain. The eyes 
receives messages then the brain has to work out the significant of these messages. 

Reading (aloud) has been considered as the way for the learners to practice their pronunciation as well as 
comprehension. Thus, reading contributes significantly to the development of the speaking skills with regards to 
pronunciation. Reading and speaking activities can be made integrated (Zhang, 2009). In addition, it is argued 
(Reading is very important in the process of teaching and learning language since it will give the learners chance 
to develop their reading strategy and acquire knowledge about many things according to their interest and 
purposes (Harmer, 2003, p. 210). Therefore, reading may provide learners with wide range of knowledge and 
reference. 
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As reading is really important for succesful university students, they must be equipped with this skill. 
Furthermore, in order to get maximum benefit from thier reading, the students need to be involved in both 
extensive and intensive reading. Dealing with extensive reading, the teacher can encourage the students to 
choose for themselves what they should read and to do for pleasure and general language improvement (Bell, 
1998). Meanwhile, on intensive reading the teacher chosen and directed and is designed to enable the students to 
develop their specific receptive skills. 

The above two controversial facts about reading, we, a team of researchers, were triggered to create a sort of 
breakthrough on how to upgrade the students’ reading interests through the development of Online Reading 
Assessment (ORA), in addition to the existing off-line reading assessments by means of which no e-books are 
left ‘unclicked’ and no textbooks in the library shelves are left untouched. 

1.2 Statement of the Problems 

Based on the background, the research questions can be formulated as (1) ‘What software application system can 
be employed to upgrade the students’ reading interests?’ and (2) ‘How are the reading materials integrated in the 
system as a means of online reading assessment to upgrade the students’ reading interests? 

In this respect, a model of online reading assessment has to be devised and its effectiveness tested in order to 
come up with a rigid model which can be used in different EFL contexts throughout at least Central Java. The 
steps employed in devising a model included (1) Exploration Phase in which internet browsing was performed to 
select one software application (among many) to be used in the current study. (2) Model Development Phase in 
which the model of online reading assessment was designed, and (3) Model Evaluation in which the model was 
tested in a narrow scope to find out its effectiveness. 

2. Method 

2.1 Research Desgn 

The current study was Research and Development (R & D) in nature. As outlined in Borg and Gall (2007), R & 
D is a process of investigation employed to develop and validate or test a product (of education). To accomplish 
the study, the researchers have, therefore, to conduct a needs analysis on which to design the product, and to test 
its effectiveness so as to be used for a more general scope (Sugiyono, 2009). 

In the current R & D, the product developed and validated to upgrade the students’ reading interests consists of 
(1) Facebook group discussion called “Reading Maniacs” (2) a paid software application into which reading 
assessments are integrated (3) a report of validation of the model to upgrade the students’ reading interests. 

2.2 Data Collection 

The techniques of data collection were internet browsing for a suitable software application, library studies and 
focused group discussion (FGD). The three techniques were collaborative in nature in order to obtain the valid 
data. Library studies were used to obtain the suitable materials for online reading assessments. Meanwhile, FGD 
(practitioner and expert judgments) was used to provide inputs for the current model of online reading 
assessments 

2.3 Research Procedures 

Modified from Borg and Gall (2007)’s ten phases, the current study was performed in seven phases, namely (1) 
research and information collecting, (2) developing a preliminary form of product, (3) preliminary field testing, 
(4) main product revision, (5) main field testing, (6) final product revision, and (7) dissemination and 
implementation. The current report is the report for research project year 2, which is more focused on the last 
two phases, to recommend the product for a wider use. 

Only very recently has the product of the research—model of online reading assessment been launched for use in 
three faculties in Stikubank University, namely the faculty of language and cultural studies, the faculty of 
economics and business studies, and the faculty of informatics. One English reading class was selected from 
each faculty to participate in the use of online reading assessment project to find out how it worked. Each student 
was supposed to join Facebook Group Discussion, referred to as “Reading Maniacs” to participate in ORA. Each 
week, a new reading material was uploaded; there were 12 weeks in one semester, thus, there were 12 materials 
all together, covering the question types, such as Multiple Choice, True-False, Matching, Sequencing, and Essay. 
Responses were automatically sent to the teacher’s email. Feedbacks for the students were set at the end of each 
session and were freely accessed by students upon completion of the test. 
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At the end of the semester, questionnaire focusing on the nature of ORA in relation to its effectiveness to 
encourage the students’ reading interests was administered to the three English reading classes (40 respondents) 
to find out the students’ perceptions of the assessment model. Interviews to selected students (6 interviewees) 
from the three faculties (of Law, Economics and Business Studies, and English Literature) were conducted to 
convince the results of the current study. 

3. Results 

3.1 Evaluation of Software Application System 

The choice of Question Writer (QW3.5) was not meant for its marketing purposes. The researcher team has no 
business connection with QW3.5 developer. It was just a purchase of the software based on careful investigation 
and evaluation of the software performance. It turned out that the software met the feature requirements as set by 
the researcher team, and outlined below: 

1) The software can accommodate various types of questions, such as multiple choices, multiple choices with 
credits, blank-filling, sequencing, matching and essay. 

2) The students’ responses are automatically scored and directly forwarded to the teacher’s email, except for 
the essay type; the scoring is left to the teacher who then can use a special rubric to score the essay. 

3) There is an unlimited space to store both the questions and the students’ responses in a program called 
“question writer tracker”. This is good for assessment of a large number of students 

4) The questions are editable at any time with or without any feedback for the correct answer. It is up to the 
teacher whether or not a feedback feature is set. 

5) The current study made use of question writer (QW3.5). It is possible for upgrade to a higher series, for 
example QW4.5. 

It should be re-stressed that the current study is by no means for promotional (marketing) of the software. Other 
software application systems may of course be used as far as the features meet the required criteria for efficiency. 
More similar software application systems may be available. The choice of QW3.5 was partly due to ease of 
operation and administration. 

3.2 Sample of Online Reading Assessments 

Below are examples of online reading assessments (True-False and Multiple Choice types). The students were 
exposed to a reading text displayed on the screen. After a defined period of time, each student with his or her 
own laptop was supposed to get access to ‘questions’ located at Facebook group discussion entitled “Reading 
Maniacs”. Of course it was possible for the students to reread the text as it was displayed during the learning and 
teaching processes. At one time or another, the text was also posted in Facebook group discussion for the 
students who might want to get access to the materials outside the reading class. 

3.2.1 True-False Type 

The students simply click on the alternative according to their comprehension about the text. To continue doing 
the text, they simply click the ‘next’ button. Then, the next question appears for the students to do. This also 
applies to the multiple choices types of test. 
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True-False 

Question 1 of 4 

According to the text you just read on the screen, US special forces were attacked by Taliban in return to the death 
of Taliban's leader. 

True 

False 

(10 points) 



Figure 1. True-False question type 


3.2.2 Multiple Choice 


CHOOSE THE BEST ANSWER 

Question 4 of 10 

There are a few steps you can talk to improve your memory for names and faces. First of all, listen carefully 
for a person’s name when he or she is introduced to you. Then repeat the name in responding to the 
introduction . If the name is unusual, ask how it is spelled. Then use the name several times to yourself and 
by visualizing his or her face in the hour after you met the person . The topic is : 


Improving memory 
Improving memory for names 
improving memory for names and places 
introduction 


(10 points) 



Figure 2. Multiple choice question type 


As outlined above, more test types, such as sequencing, matching, and essay can be made on Question Writer 
software (CW.3.5). For matching and sequencing, dragging one item to match or sequence with the “suitable 
pairs” is required of the students in order to do the test. Meanwhile, for essay test, the students will have to type 
the responses on the spaces provided. A specially designed rubric must be devised to score the essay type test. 

The students’ responses were automatically scored (except for essay) and sent to the teacher’s email as shown in 
the setting below: 

3.2.3 Properties Window 

ORA’s properties can be set as required by the teacher for efficiency purposes. 
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Properties Window 


Quiz Details 
Introduction 


Question Options 

A ppearance Settings 

risi Settings 


Metadata (SCORM) 
Report Settings 
Report Labels 


Other Labels 


Send Results Over The Internet 

0 Send Results Via Websie To E-mail (Best Effort Service) 
endangyufcam @gmaJ com 


□ Send Results Before Showing Any Feedback 
I I Use a Custom Results Server 

http ://www questionwntertracker .com/colat or/results php 7 emal= 


I I Use a Custom Results Backup Server 

http ://www xquestion com/resuls/resulsqw3 php^email * 


Figure 3. Properties setting 


From Figure 3, it is clear that the teacher can set ‘the properties’ of the software, one of which is to send the 
results (of the test) to the teacher’s email in addition to the other properties (1) sending result before showing any 
feedback, (2) using a custom result server—for a large number of students, and (3) using a custom result backup 
server. 

In addition, to minimize cheating among the students, all question items and the alternatives can be randomized. 
Individual student would get access to a different format of the test. Below is an example of setting 
(randomized). 


0ui2 Details 
Introduction 
Question Options 
Appearance Settings 
Text Settings 
Web Settings 
Metadata (SCORM) 
Report Settings 
Report Labels 
Other Labels 


General Question Properties 
0 Show Question Number 


0 Randomize Question Options 


0 Display Point Values For Questions 
0 Display Title With Each Question 


n Use Image Zooming 


I I Never Show Back Button With Questions 


U Force Attempt at Every Question 


Figure 4. More settings 


From Figure 4, it is clear that more possibilities for settings, such as (1) show question number, (2) randomize 
question option, (3) display Title with each question, (4) use image zooming, (5) never show back button with 
questions, and (6) force attempt at every question. 

3.2.4 Feedback 

Feedback to each student is shown at the end of the test. It depends on the teacher’s policy for the students 
whether they have to retake the test or not upon failure. 
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ular 


Multiple Choices 

mary Report 


Your Score: 

Your Score (percentage) 

30 

30 % 

Total Possible Score (poirts) 

Passing Score: 7S 

100 

£ 

O 

00 

Quiz Candidate 


W 


Test Status* 

Fail 


Message 

Failed 



Figure 5. Student’s Feedback 


From Figure 5, it is clear that the student failed. With only 30%, he or she was not able to reach the minimum 
overall score of 80% with the passing score of 75. Therefore the test status is Fail; Message= Failed. 

3.3 Students’ Perceptions in the Use of Online Reading Assessments 
3.3.1 Questionnaire-Based 

The questionnaire consisted of fifteen statements regarding the online reading assessments (ORA). There are five 
options for the students to choose, (1) strongly disagree, (2) disagree, (3) neutral, (4) agree and (5) strongly agree. 
The results of the first five statements are tabulated below: 
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Table 1. Questionnaire results 

No 

Statements 

5 

4 

3 

2 1 

I 



% 

% 

% 

% % 

% 

1 

Online Reading Assessment (ORA) supports the new paradigm of 
language testing 

16 

84 



100 

2 

Computer literacy with respect to the administration of ORA is at the 
appropriate level. 


65 

30 

5 


3 

Quick scoring and direct feedback are good features of ORA. 

11 

89 




4 

Personally, I was very much at ease doing the test without technical 
problems. 


87 

13 



5 

Test validity and reliability are questionable since ORA may be done 
by someone else. 



76 

24 



5=strongly agree; 4=agree; 3=neutral; 2=disagree; 1 -strongly disagree. 


From Table 1 above, it is clear that ORA supports or is in line with the new paradigm of language teaching. The 
respondents (16%) expressed strong agreement, while the other group (84%) agreed to the statement. Even, it is 
argued that later on paper-based TOEFL would soon be replaced by TOEFL iBT. Therefore, it is a good idea to 
start with Online Reading Assessment (ORA) as starting exercises. 

With respect to computer literacy required in order to be able to do ORA, 65% of the respondents felt that they 
were appropriately equipped. This is logical since almost everyone is computer literate at least at users level. 
Other respondents (30%) stayed neutral, representing the fact that they might still need practicing in doing ORA. 
Still others (5%) expressed disagreement. This group of respondents might have problems doing ORA, such as 
dealing with slow internet access and possible technical constraints. From this point, it is clear that doing online 
tests not only requires the ability to do the test ‘materially’ but also a certain level of computer literacy. A 
particular individual may get ‘failed’ without possessing sufficient computer literacy even though he or she may 
have reached a considerable degree of language proficiency. 

Strong agreements were expressed by the respondents (11%) with respect to the fact that quick scoring and direct 
feedback were competitive advantage over the off-line (paper-based) test. The other respondents (89%) further 
strengthened the fact. There are of course so many advantages of implementing ORA that, later on, regarding 
language proficiency tests, online testing systems would take over off-line testing systems in accordance with the 
advanced development of information technology. 

With respect to the perception of each individual in doing ORA, 87% of the respondents felt at ease and the other 
13% remained neutral. It is understandable that the students in terms of their computer literacy mastery, found it 
easy technically to do ORA. 

As an assessment, the use of ORA remained questionable regarding the results of the test, whether it has 
achieved the goal of the test—to locate the students within an appropriate level of proficiency. This was proved 
by 58% respondents who agreed with the statement that ‘Test validity and reliability are questionable since ORA 
may be done by someone else’. The other respondents (42%) remained neutral. It all has something to do with 
the students’ character building—to remain honest. 

Tabulated below are the results of the next five statements of the questionnaire. 
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Table 2. Questionnaire results 

No 

Statements 

5 

4 

3 

2 

1 

I 



% 

% 

% 

% 

% 

% 

6 

ORA must be done at the same time by the students 



33 

67 


100 

7 

Special rubric must be established for essay-scoring 

78 

22 





8 

ORA may be done off campus. 




89 

11 


9 

ORA is not recommended for final scoring and grading. 



21 

79 



10 

ORA is best recommended for supplementary use. 


72 

28 





5=strongly agree; 4=agree; 3-neutral; 2=disagree; I -strongly disagree. 


One advantage of using ORA is that the test may be done at different times by different students. It is therefore 
logical that 67% of the respondents did not agree that ORA was done at the same time. The other 33% of the 
respondents remained neutral. The respondents who were neutral might think that ORA was flexible with regards 
to its administration. 

One constraint of ORA is that it cannot score ‘essay type test directly. The students’ essays were sent to the 
teacher’s email but remained unscored; therefore most of the respondents (78%) totally agreed that special rubric 
had to be devised to handle essay-type test. The other 22% of the respondents agreed to this. 

Similar to Statement 6, ORA may be done off-campus. However, 89% of the students remained neutral with 11% 
disagreement. It is possible that the students may ask other people to do the ORA on their behalf. This might be 
considered as the disadvantages of ORA. It is also related to Statement 9 that ORA is not recommended for final 
scoring and grading. Only 21% agreed to this with the other 79% at neutral position. It can therefore be inferred 
that ORA at the moment is best used as a supplementary testing. It was supported by 72% agreement and 28% at 
neutral position. 

Tabulated below are the results of the final five statements of the questionnaire. 


Table 2. Questionnaire results 


No 

Statements 

5 

4 

3 2 

1 x 



% 

% 

% % 

% % 

11 

Both items and alternatives can be rendomized, indicating good 
features of ORA. 

10 

90 


100 

12 

ORA’s time-based feature is similar to TOEFL iBT 


92 

8 


13 

It is recommended ORA be used in place of paper-based test 


84 

16 


14 

The students’ reading interests increase in response to the use of 
ORA. 

21 

79 



15 

ORA is recommended as it encourages the students themselves to 
equip themselves with sufficient computer literacy level 

11 

89 




5=strongly agree; 4=agree; 3-neutral; 2=disagree; l=strongly disagree. 


Another competitive advantage of ORA is that both question items and alternatives can be rendomized to avoid 
the same forms of questions done by different students. This feature was considered good by 10% total 
agreement and 90% agreements. In this case, cheating can be somehow minimized. 

ORA is time-based test and is therefore similar to TOEFL iBT. Therefore, ORA somehow helps the students get 
familiar with TOEFL iBT. Most of the respondents (92%) agreed to this with only 8% at neutral position. 
Furthermore, 84% of the respondents agreed to the recommendation that ORA replace the paper-based test of 
reading comprehension; 16% of the respondents remained neutral. 
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The aim of the test to increase the reading interests of the students is somehow successful as 21% of the 
respondents totally agreed that ORA increased the reading interests (the other 79% of the respondents agreed). 
This is also an indicator that most of the students are computer literate. Those students with sufficient computer 
literacy felt very much encouraged to do ORA as supported by 11% total agreements and 89% agreements—with 
respect to level of computer literacy. 

3.3.2 Interviews-Based Perceptions 

As outlined above (Method), six students were picked up for interview to convince the results of the 
questionnaire. Most of the students responded positively to ORA as a supplementary test in addition to the 
traditional off-line reading assessments. Only one student maintained that honesty was still questionable since 
such an online test could be done by someone else—to be warned that Indonesian students were very cooperative, 
negatively including doing exams for other people. Most importantly, ORA was agreed to be able to increase the 
reading interests—strongly supporting Statement 14 in the Questionnaire. 

All agreed though that ORA was a good exercise for the students since they may at one time or another take 
TOEFL iBT where everything is done online. Even it is assumed that paper-based TOEFL will be put to its end, 
meaning that in the future, TOEFL will be internet-based. 

The interviewees also expressed satisfaction with respect to the administration of ORA, including the use of 
Facebook group discussion as a medium of interaction between the teacher and the students in addition to 
between students and students. It was also agreed that the testing materials had to be varied, making the best use 
of all facilities that the software offered. Updates should be done on a regular basis to comply with the demand 
for modern testing, evaluation and assessment. 

4. Conclusion and Recommendations 

Much has been discussed on the implementation of software application system for the development of language 
skills, such as reading, which is thought of as the mother of learning. Without reading it is hard to imagine how 
knowledge can be preserved and shared. The integration of online reading assessments (ORA) into a software 
application turns out to be well appreciated by the students as a means of upgrading their reading interests. This 
is in line with the current era of cyber technology. Furthermore, online reading assessments can be accessed via 
desktop computer, laptop, gadgets or even smart mobile phones. Thus, it really encourages freedom of modes of 
teaching and learning. In other words, interaction between the teacher and the students does not have to be 
face-to-face interactional mode. Another good point with the use of online reading assessments is that busy 
students—those who are also working—can still get access to the assessments within a defined period of time. In 
other words, they do not have to take the test at the same time. 

It is therefore recommended (1) that teachers develop other online assessments in relation to the improvement of 
other language skills (2) that the students’ participation in such a type of assessments must be based on supreme 
honesty since it is possible that this kind of assessment, due to the online nature, can be done or performed by 
other individuals for and on behalf of particular students. For this, it is necessary that the teacher set a time limit, 
for example, the test should be done on a specific date, at specific time. Still another recommendation of 
relevance, considering that the questionnaire was only administered to the students, is that future researchers 
may want to administer specially designed questionnaire to the reading teachers to obtain their feedbacks 
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